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Foreword 


The National Institute of Standards and Technology (NIST) develops and promotes 
measurement, standards, and technology to enhance productivity, facilitate trade, and 
improve quality of life. In the aftermath of the attacks of September 11, 2001, NIST has 
taken a key role in enhancing the nation’s homeland security. Through projects spanning 
a wide range of research areas, NIST is helping millions of individuals in law 
enforcement, the military, emergency services, information technology, the construction 
industry, and other areas protect the American public from terrorist threats. 


NIST’s Building and Fire Research Laboratory (BFRL) has as its mission to meet the 
measurement and standards needs of the building and fire safety communities. A key 
element of that mission is BFRL’s commitment to homeland security. Specifically, the 
goal of BFRL’s homeland security effort 1s to develop and implement the standards, 
technology, and practices needed for cost-effective improvements to the safety and 
security of buildings and building occupants, including evacuation, emergency response 
procedures, and threat mitigation. The strategy to meet this goal is supported by BFRL’s: 


e research and development (R&D) program to provide a technical foundation that 
supports improvements to building and fire codes, standards, and practices that 
reduce the impact of extreme threats to the safety of buildings, their occupants 
and emergency responders; and 


e dissemination and technical assistance program (DTAP) to engage leaders of the 
construction and building community 1n implementing proposed changes to 
practices, standards, and codes. DTAP will also provide practical guidance and 
tools to better prepare facility owners, contractors, architects, engineers, 
emergency responders, and regulatory authorities to respond to future disasters. 


This report, prepared for NIST by the Construction Industry Institute (CII), was funded as 
part of BFRL’s homeland security R&D program. It complements previous DTAP- 
funded research into best practices for project security by documenting key lessons 
learned from implementing these practices. The security best practices were developed 
by CII and published in 2004 as NIST GCR 04-865. This report includes a discussion of 
baseline measures of performance, information on a set of case study projects, a summary 
of significant findings producing the lessons learned, and how to use the lessons learned 
to improve implementation for future projects. The case study projects provided 
significant supporting information confirming a project team’s ability to apply the 
security practices. Based on in-depth reviews of the case study projects, CH concluded 
that: (1) the security practices provide project teams access to a flexible and structured 
framework for the integration of security into the project delivery process (2) project 
teams experienced the most success when they incorporated implementation steps into 
existing organizational processes and procedures. 


The material presented in this report complements research being conducted by the 
Office of Applied Economics (OAE) under BFRL’s homeland security R&D program. 


ill 


OAE’s research focuses on developing economic tools to aid facility owners and 
managers in the selection of cost-effective strategies that respond to natural and man- 
made hazards. OAE’s research has produced a three-step protocol for developing a risk 
mitigation plan for optimizing protection of constructed facilities. This protocol helps 
decision makers assess the risk of their facility to damages from natural and man-made 
hazards; identify engineering, management, and financial strategies for abating the risk of 
damages; and use standardized economic evaluation methods to select the most cost- 
effective combination of risk mitigation strategies to protect their facility. This report 
covers key components of the first two steps of the three-step protocol. 


Robert E. Chapman 

Office of Applied Economics 

Building and Fire Research Laboratory 
National Institute of Standards and Technology 
Gaithersburg, MD 20899-8603 
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Abstract 


This research was performed by the Construction Industry Institute (CII) and 
complements previous research into best practices for project security. The security best 
practices were developed by CII and published as NIST GCR 04-865; however, little data 
still exist on the implementation of these practices. This lack of information impeded 
utilization of the practices due to uncertainties about cost and schedule impacts as well as 
impacts on established project processes. This research addresses this lack of 
information. 


The research consisted of three tasks. The first task included a statistical analysis of a 
broad cross section of projects to produce baseline measures of performance used to 
establish a desired set of characteristics typical of exemplary projects. The second task 
entailed the selection of a set of exemplary projects for case-by-case analyses. The case- 
by-case analyses included observation of project team implementations using facilitated 
discussions to collect anecdotal information resulting in lessons learned. The final task of 
the research was a synthesis of the findings. This synthesis includes a discussion of 
baseline measures of performance, a summary of significant findings producing the 
lessons learned, and how to use the lessons learned to improve implementation for future 
projects. 


The case study projects provided significant supporting information confirming a project 
team’s ability to apply the security practices. The security practices provide project 
teams access to a flexible and structured framework for integrating security into the 
project delivery process. Project teams experienced the most success when they 
incorporated implementation steps into existing organizational processes and procedures. 


Keywords 


Best practices, chemical manufacturing, energy production and distribution, facility 
security, industrial facilities, security rating index, and transportation infrastructure 
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1. INTRODUCTION 


The events of 9/11 have had far reaching impacts on the country and have heightened the 
security awareness of industry leaders responsible for critical infrastructure facilities. 
The increased awareness has led to numerous security enhancing initiatives. Research 
funded by the Demonstration and Technical Assistance Program of the Building and Fire 
Research Laboratory (BFRL) at the National Institute of Standards and Technology 
(NIST) and published as NIST GCR 04-865 identified best practices for project security. 
The Construction Industry Institute (CII) working with NIST developed these practices 
and a database for collecting data on their implementation. However, little data still exist 
on the implementation of these practices. This lack of information impedes utilization of 
the practices due to uncertainties about cost and schedule impacts as well as impacts on 
established project processes. This research was funded as part of BFRL’s homeland 
security R&D program and performed by CII at the University of Texas at Austin. It 
complements the previous efforts by developing a comprehensive set of information 
based on actual project execution experiences that documents key lessons learned from 
implementing best practices for project security. 


The material presented in this report also complements research being conducted by the 
Office of Applied Economics (OAE) under BFRL’s homeland security R&D program. 
OAE’s research focuses on developing economic tools to aid facility owners and 
managers in the selection of cost-effective strategies that respond to natural and man- 
made hazards. OAE’s research has produced a three-step protocol for developing a risk 
mitigation plan for optimizing protection of constructed facilities (Chapman and Leng, 
2004). The protocol helps decision makers assess the risk of their facility to damages 
from natural and man-made hazards; identify engineering, management, and financial 
strategies for abating the risk of damages; and use standardized economic evaluation 
methods to select the most cost-effective combination of risk mitigation strategies to 
protect their facility. This report covers key components of the first two steps of the 
three-step protocol. 


1.1 Background 


The earlier research funded by NIST and completed by CII in 2004, identified best 
practices for project security and a methodology for scoring the level of implementation 
of these practices. In a follow-on effort, CII developed a framework for integrating these 
practices into the project delivery process which was published in 2005 as a CII 
Implementation Resource IR BMM-3, Implementing Project Security Practices. Based 
on the input of industry experts, the implementation resource defines in detail a process 
for integrating the security practices into all phases of project planning and execution and 
suggests that this process will enable more cost effective security throughout the project 
lifecycle. The lack of actual project data on use of the practices and impacts on 
established project processes; however, hinders their acceptance. 


CII also maintains a benchmarking database for capital facility projects containing 
approximately 1 420 projects representing over $66 billion in total installed cost. More 


than 700 projects in this database fall into the sectors of chemical manufacturing, energy 
production and distribution, and transportation infrastructure. The projects drawn from 
the three sectors include new construction, additions, and modernization projects, ranging 
in cost from less than $5 million to in excess of $100 million. The richness of the CII 
database provides an excellent data set for the selection of project specific characteristics 
representative of exemplary projects. These characteristics were desired in the selection 
of projects for a case-by-case analysis to fill the information gap on implementation of 
security practices. 


1.2 Purpose 


The purpose of this research was to produce a comprehensive set of information based on 
actual project execution experiences derived from implementing the best practices for 
project security. Specifically, this investigation documents the level of security practice 
implementation during the project delivery process using a nine-step procedure defined in 
the CII implementation resource IR BMM-3 and described in Section 3.3 of this report. 
In addition to recording a project team’s ability to implement the practices, this research 
documents lessons learned for better integrating the security practices into the project 
delivery process. 


1.3. Scope and Approach 


The scope of this research consisted of three distinct tasks, all limited to the chemical 
manufacturing, energy production and distribution (to include oil production and refining, 
natural gas processing and distribution, and power generation and distribution), and 
transportation infrastructure sectors. The first task included a statistical analysis of a 
broad-cross section of projects within the limits of these sectors. This analysis produced 
baseline measures of performance used to establish a desired set of characteristics typical 
of exemplary projects. These characteristics were drawn from industry norms on five key 
outcomes: cost, schedule, safety, project development and scope changes, and field 
rework. 


The second task, using the characteristics identified from the Task | data analysis, 
entailed the selection of a set of exemplary projects for a case-by-case analysis. The task 
involved presentations to the project teams on the security best practices and the 
proposed method for their implementation. Subsequent analyses included observation of 
project team implementations using facilitated discussions to collect anecdotal 
information resulting in a series of lessons learned. The lessons learned captured include 
issues and difficulties with implementation efforts, perceived benefits for both completed 
and ongoing projects, impacts on implementation caused by the facilitation process, and 
implementation barriers. 


All selected projects were domestic; however, each was located in differing regions 
within the country ranging from the East Coast and Southeast to the Midwest and Gulf 
Coast areas of the nation. The geographic dispersion of the projects afforded the 


opportunity to observe security practice implementation under varying circumstances and 
against a range of potential threats. 


The final task of the research was the documentation of the results from Task | and Task 
2 in this technical report. The report synthesizes the findings from both the statistical 
analysis and the case-by-case analysis of the selected projects. This synthesis includes a 
discussion of baseline measures of performance, a summary of significant findings 
producing the lessons learned, and how to use the lessons learned to improve 
implementation for future projects. 
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2. STATISTICAL ANALYSIS 


This section provides a summary of the statistical analysis of a broad cross-section of 
projects drawn from the chemical manufacturing, energy production and distribution, and 
transportation infrastructure sectors. A brief description of the data set is presented along 
with performance outcomes for both owner and contractor organizations. Finally, an 
overview of the security practices 1s provided in addition to a means to measure the 
degree of implementation of these practices. 


2.1 Description of Data Set 


A total of 723 projects meeting the criteria specified by NIST were used for the analysis. 
Data were first separated for owners and contractors and further categorized by project 
type, project size and project nature. The three industry groups specified above 
(chemical manufacturing, energy production and distribution, and transportation 
infrastructure sectors) were separated into three distinct project cost categories: those 
projects with a total installed cost of less than $15 million, those between $15 million and 
$50 million, and those greater than $50 million. Each project was also categorized by 
project nature as an addition, grass root, or modernization. Figures 2-1 to 2-4 provide 
distributions of the number of owner and contractor data by respondent type, project type, 
project size, and project nature. 


Figure 2-1. Data Set by Respondent 
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Figure 2-2. Data Set by Project Type 
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Figure 2-3. Data Set by Project Size 
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Subject to the selection criteria discussed above, all projects in the CII database were 
eligible for inclusion in this analysis. In some cases though, item responses were 
excluded from the detailed analyses that follow because they were deemed to be 
statistical outliers based on the decision rule described in Appendix A. The number of 
projects represented in the tables was further reduced by item nonresponse and 
confidentiality rules which are also explained in Appendix A. Thus, while the data in 
Figures 2-1 through 2-4 above represent all the projects included in the analytic data set, 
the data included in Tables 2-1 and 2-2 represent only those data values that are more 
typical values throughout the entire distribution of projects. 


All data are presented as aggregates to ensure that no individual project could be 
identified in any of the charts or tables. When the risk of identifying a project in any 
subcategory was increased due to the small number of projects in that subcategory, the 
data for that subcategory are suppressed to ensure confidentiality. This was particularly 
true for analyses of transportation infrastructure projects and change performance metrics 
across most project types. For these cases, data are not shown within Tables 2-1 and 2-2 
due to the CII Confidentiality Policy and are identified with the code “C.T.” 
(confidentiality test). 


Tables 2-1 and 2-2 summarize the key project outcomes for owner and contractor 
projects respectively. In each summary, mean values are provided to establish industry 
norms for five categories: cost, schedule, safety, changes, and field rework. To assist in 
the analysis of the table data, the best performance in each category 1s shaded. Appendix 
B contains metric definitions and formulas for each measure used within the table in 
addition to project phase definitions. 


2.2 Project Outcomes - Owners 


Within the key outcomes of cost and schedule, owner respondent projects achieved an 
overall project cost growth of -4.9 percent with a mean schedule growth of 4.9 percent. 
With respect to safety, owner projects reported a recordable incidence rate of 1.437 anda 
remarkable lost workday case incidence rate of 0. While the lost workday case rate is 
hard to comprehend on a sample this size, verification of the data reveals that any lost 
workday incidents are statistical outliers which were stripped from the sample by the data 
analysis rules documented in Appendix A. 


In general, energy production and distribution projects experienced somewhat better 
overall project cost performance compared to chemical manufacturing projects as shown 
in the shaded cells of Table 2-1; however, the chemical manufacturing projects reported 
stronger schedule performance than their energy counterparts. Insufficient data are 
available to report for transportation infrastructure projects. 


Within cost categories, larger projects greater than $50 million reported better project and 
construction cost growth. This particular outcome is consistent with CII previous 
experience due to larger projects typically reporting a greater use of performance 
enhancing best practices. Overall schedule growth though, was better in the mid-sized 
$15 million to $50 million category with the less than $15 million projects excelling at 
construction schedule growth. 


With respect to project nature, as expected grass roots projects reported the best project 
cost and project schedule growth in comparison to addition and modernization projects. 
Although grass root projects tend to be larger in size, they generally experience less 
complexities and interferences common in the execution of addition and modernization 
projects. 
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2.3 Project Outcomes - Contractors 


In general, contractor respondents reported greater cost growth in comparison to owners; 
however, this 1s likely reflective of the competitive nature of the bidding process resulting 
in lean budget estimates and differences in the treatment of changes between owners and 
contractors. The overall project cost growth of 4.4 percent is 9.3 percentage points 
higher than the owner counterparts. In contrast, contractors experienced a project 
schedule growth of only 2.8 percent; over two percentage points better than owners. This 
likely is reflective of a lesser role in the drawn out planning and start-up phases. 
Contractors also reported slightly higher safety recordable rates and much higher lost 
workday case incidence rates, although, these rates are still very good in comparison to 
Bureau of Labor Statistics (BLS) published norms. 


By project type, the energy production and distribution sector performed best for most 
cost, schedule, and safety metrics. It is interesting to note that the larger cost category of 
projects for contractors underperformed the others in every measurable key outcome, 
although this finding likely lacks statistical significance. This differs from the owner data 
where larger projects experienced better project cost growth. Also of interest for the 
contractors is the better performance of the modernization projects on cost and safety 
metrics, although in most cases the differences lack any practical significance. 
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3. SECURITY BEST PRACTICES AND MEASUREMENT 
OF THEIR LEVEL OF IMPLEMENTATION 


The statistical analyses in Section 2 provide characteristics for exemplary projects for 
chemical manufacturing, energy production and distribution, and transportation 
infrastructure based on the calculation of performance norms. Knowledge of the 
performance of these projects would be useful as additional projects were selected for the 
case-by-case analysis of security practice implementation required in Task 2 to produce 
lessons learned. 


Capitalizing on the expertise and knowledge base of CII with respect to best practices 
within the construction industry, NIST contracted with CII to develop security best 
practices and a quantifiable measure of their integration in the capital facilities delivery 
process, which became known as the Security Rating Index (SRI). This section provides 
an overview of the security best practices and the SRI which were published in 2004 as 
NIST GCR 04-865 (Thomas, et al., 2004). A process for implementation of these 
practices was subsequently developed by CII and published in 2005 as an Implementation 
Resource, IR BMM-3, Implementing Project Security Practices (CII, 2005). 


3.1 Security Best Practices Overview 


The joint NIST and CII study identified 33 security practices for implementation during 
front-end planning through project start-up. In general, the practices identified project 
activities in which security should be considered and perhaps integrated based on the 
perceived threat and the worst case consequence of a security breach. Methodology for 
assessing the level of threat and consequence is discussed in NIST GCR 04-865 and CII 
IR BMM-3. While specific details of security implementation will be particular to the 
project at hand, the activities identified 1n the study provide the project team a 
compendium of practices that should lead to improved security by raising awareness of 
the need for security integration at critical stages of project planning and execution. 


Table 3-1 lists all 33 practices, the project phase in which they apply, and the security 
elements which they address. Project phases are defined in Appendix B and security 
elements are defined following the table. As depicted in the Table 3.1 below, most 
practices are applicable to multiple project phases and often address more than one 
security element. A review of the table confirms that most practices are front-end loaded 
and the relative importance of the practices likely varies considerably. The original NIST 
and CII research determined through industry experts that activities occurring earlier in 
the project should carry more weight in most cases than activities occurring later in the 
project. A thorough discussion of the process for establishing the weights is provided in 
NIST GCR 04-865. 


Table 3-1. The Security Best Practices 


Security Element’ 


Establishing project objectives (e.g., reliability and operating 
philosophy, affordability and feasibility, constructability, future 


Preparation of the specifications and requirements (e.g., 
civil/structural, architectural, water treatment, 
loading/unloading/storage facilities, substation/power sources, 
instrument & electrical, etc.) 


Developing and evaluating design criteria (based on vulnerability 
assessment) 


[a foowtpnsericsone Cd x 
ee a 
[8 [Developing the engineeringloonstructon plan & approach |X |X a.hlU= 


Developing the procurement/materials management procedures 
and plans (e.g., warehousing, inventory control, key & lock control, 
hazardous materials) 
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Developing the pre-comm/turnover sequence/startup 
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Developing the plot plan (i.e., layout, accessibility, gate 
configuration, etc.) - retrofit & greenfield 


Evaluation of various personnel issues (e.g., education/training, 
safety and health considerations) 


Development of a distribution matrix for document control (e.g., 
drawings, project correspondence, CAD, as-built documents) 


The project team was in alignment concerning the importance of 
security issues identified in the project objectives. 


Security-related equipment was defined and purchased with 
appropriate input (e.g., O&M, Security Manager, etc.) 


Identifying stakeholders for the project team (based on vulnerability 
assessment) 


Security Element? 


rs rep aes ea 
Establishing priorities between cost, schedule, and required project 
features (based on vulnerability assessment) 


Activity in which security should be considered: 


Identifying and resourcing startup requirements (e.g., procurement, 
personnel, training) 


Screening of the project team for appropriate level of clearance ee :f 
Screening of contractor/subcontractor employees/delivery 
ee el for snes level of clearance 
envi Slat rk re oe cs a 
eieg pe startup security plan ee x el x fae ca Ea [er] 


Developing system startup plan (reconciled with security plan) —6hw OU BE 
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Developing/implementing construction site security plan (e.g., fire 
28 |protection and safety considerations, egress, emergency responder X X X X X 
access, process shutdown) 
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31 Project information systems security plan (e.g., firewalls, wireless 
security, passwords, access controls) 
Security breaches/incidents were routinely investigated Sanaa | 


33 Developing emergency response plan in coordination with local X Xx 
authorities 


‘Phase definitions are provided in Appendix B. Phase abbreviations are as follows: 


e FEP: Front End Planning 
e D: Design 

e P: Procurement 

e CON: Construction 

e SU: Start-Up 


*Security Elements Definitions (Center for Chemical Process Safety, 2002): 
e Physical security considerations include equipment, building and grounds design, and security 
practices designed to prevent physical attacks on facilities, persons, property, or information. 
e Personnel security includes practices and procedures for hiring, terminating, and addressing 
workplace issues; screening or background checks of employees. 
e Information security refers to practices and procedures for protection of documents, data, 
networks, computer facilities, and telephonic or other verbal communication. 
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3.2 Measuring the Degree of Security Practice Implementation 


The Security Rating Index published in NIST GCR 04-865 provides a means to 
quantitatively assess the level of security practice implementation for a specific project. 
Project teams assess the level of implementation by answering questions concerning the 
degree that security was considered while performing typical project tasks. There are 33 
questions, one for each security practice; however, questions may be asked multiple times 
as most of the practices are relevant to multiple project phases. The questions are framed 
by the statement that “Security was a consideration in...” followed by a specific project 
task pertinent to the project phase. Figure 3-1 provides sample questions and the six 
possible responses ranging from Strongly Disagree to Strongly Agree in addition to a 
non-applicable / unknown response. A complete list of the 33 security practices in 
questionnaire format is provided in Appendix C. 


Figure 3-1. Security Questionnaire Example 


Strongly Disagree 
Disagree 
Strongly Agree 
NA/Unknown 


_______Securitywasaconsiderationin 
establishing project objectives (e.g., reliability and | 
operating philosophy, affordability and feasibility, 
constructability, future expansion, etc.). 


preparation of the specifications and requirements (e.g., | 
civil/structural, architectural, water treatment, 
loading/unloading/storage facilities, substation/power 
sources, instrument & electrical, etc.). 


developing and evaluating design criteria (based on — 
vulnerability assessment). 


Each security question was weighted using the analytical hierarchy process (AHP) to 
better reflect its relative importance for security based on the opinion of industry experts. 
The resulting weights from this process provided the foundation for the SRI scoring 
algorithm used to calculate a Phase SRI score. The algorithm calculates an overall 
project score that assesses practice implementation on a scale of 0 to 10, with a higher 
score representing a higher level of implementation. A detailed description of the scoring 
algorithm and the method of calculating the SRI score can be found in NIST GCR 04- 
865. 


As discussed in NIST GCR 04-865, SRI scores must be viewed in the context of threats 


and consequences of potential security breaches (Thomas, et al., 2004). Projects with 
differing characteristics, such as location, infrastructure criticality, and potential 
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environmental impacts have differences in adversarial threat and consequences of a 
security breach and thus, require differing levels of security practice implementation. 
Determination of project threat and consequence levels therefore, provides a common 
basis for comparing levels of security implementation among projects. Table 3-2 and 
Table 3-3 depict the Threat and Consequence Level assessment matrices for project team 
use developed by the original NIST and CII research. Use of these matrices is fully 
discussed in NIST GCR 04-865. 


As depicted in Table 3-2, the threat ratings range from Very Low to Very High on a scale 
from | to 5. Project teams with the assistance of company security and risk assessment 
professionals evaluate their project threat level based on a potential adversary’s capability 
to conduct an attack, their intent to attack the project, and historical events identifying 
cases where similar facilities have been attacked. 


Table 3-2. Threat Levels 


5: Very High 


Threat Rating ‘lg Description 


Indicates that a definite threat exists against the asset and that the 


adversary has both the capability and intent to launch an attack or 
commit a criminal act, and that the subject or similar assets are targeted 
on a frequently recurring basis. 


knowledge of the adversary’s capability and intent to attack or commit a 
| criminal act against the asset, based on related incidents having taken 
place at similar assets or in similar situations. 


| 
| 
| 
| Indicates that a credible threat exists against the asset based on 
| 
| 
| 


-3-Medium | Indicates that there is a s a possible threat to the asset based on the — 
adversary’s desire to compromise similar assets and/or the possibility tha 
the adversary could obtain the capability through a third party who has 


Indicates that there is a Jow threat against the asset or similar assets and 
ates __| that few known adversaries would pose a threat to the asset. | 
fe Very Low | Indicates no credible evidence of capability or intent and no history or 
| actual or planned threats against the asset or similar assets. 


| 
| 

_| demonstrated the capability in related incidents. _ 
| 


The Consequence Levels depicted in Table 3-3 quantify the worst-case outcome of a 
successful attack on the project. The scores, similar to the Threat Levels, range from 
Very Low (1) to Very High (5) and again depend on the project team’s assessment of 1) 
the types of injuries that would occur, 2) the potential environmental impact, 3) resulting 
property damage, and 4) likely business interruptions. As with the threat assessment, 
project teams should obtain input from company security and risk management personnel 
when making this assessment. 


lg 


Following the threat and consequences assessment for the project, an appraisal is made of 
the security practice use using the questionnaire in Appendix C to obtain the SRI score. 
The SRI questionnaire, in addition to providing a quantitative assessment of practice use, 
provides project teams a checklist to facilitate integration of security into project tasks 
according to typical project phases. Once sufficient data are available, norms can be 
established to compare projects with similar threat and consequence levels (Thomas, et 
al., 2004). 


Table 3-3. Consequence Levels 


Consequence Sk 
Category _| Description 


5 — Very Severe Possibility of any offsite fatalities; possibility for multiple onsite 
fatalities 
Extensive environmental impact onsite and/or offsite 
Extensive property damage 
Very long term business interruption/expense 
Possibility of any offsite injuries; possibility for onsite fatalities 


Significant environmental impact onsite and/or offsite 
Significant property damage 

Long term business interruption/expense 

No offsite injuries; possibility for widespread onsite injuries 
Moderate environmental impact onsite and/or offsite 
Moderate property damage 

Medium term business interruption/expense 
Possibility for onsite injuries 

Minor environmental impact only 

Minor property damage 

Short term business interruption/expense 

Possibility for minor onsite injuries 

No environmental impact 

Little to no property damage 

Little to no business interruption/expense 


1 — Very Minor 


3.3. Implementation Process for the Security Practices 


The SRI can be an effective means of assessing security implementation; however, it is 
often difficult for project teams to adequately use the tool without a structured 
implementation process. Recognizing this barrier, CII published a nine-step 
implementation process to assist project teams with implementation of the security 
practices and use of the SRI. This section provides an overview of the nine-step process. 


The nine steps in Figure 3-2, reprinted from IR BMM-3, provide a structured process to 
integrate security into each phase of the project delivery process. For a detailed 
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description of the implementation process, to include examples of the phase checklist and 
activity risk matrix, see Chapter 3 of IR BMM-3. 


Figure 3-2. Security Practice Implementation Process 


1. Review phase checklist 
before phase start 


2. Develop activity risk 
matrix 


3. Identify security 
practices relevant to 
project phase 


4. Irplement practices as 
appropriate 


6. Complete questionnaire 
& calculate phase SRI 
score 


6. Conduct periodic revie 


?. Update phase SRI 
score 


Yes 


8. Conduct post phase 
implementation review 


9. Closeout phase SRI 


Me 


In Step 1, “Review phase checklist before phase start”, project teams review the activities 
in Table 3-1 for the upcoming project phase to gain an understanding of the security 
practices relevant to their project for the phase which they are entering. As part of 
developing the activity risk matrix in Step 2, the project team must identify applicable 
risks to the project, classification of risks by security element, and the means of 
mitigating the risk. Step 2 also includes the project team’s initial assessment of the 
project threat and consequence level. A detailed activity risk matrix enables the team to 
better identify the security practices in Step 3 that are applicable to the upcoming project 
phase. Identification of the applicable practices enables the project team to focus on 
implementing in Step 4, only those practices that are essential for the risks given their 
threat and consequence assessments. Step 5, “Complete questionnaire and calculate 
phase SRI score”, provides the team with an initial assessment of the level of security 
implementation and should also give them a means to compare their level of 
implementation to other projects once a database of SRI scores is developed. Completion 
of the questionnaire and scoring of the SRI is accomplished using CII’s web-based data 
collection and scoring tool known as the SRI Tool. 


Since project phases can last many months or even years, periodic review and update of 
the threat and security practice use are encouraged in Steps 6 and 7. The purpose of the 
review is to identify changes in the threat and activity risk matrix and update security 
practice use as appropriate. Periodic reviews also provide the team with an opportunity 
to validate initial responses to the security questionnaire by updating practice use 
assessments to reflect changes in implementation. Upon completion of a project phase, 
the team conducts a post-phase implementation review in Step 8 to identify lessons 
learned from the completed phase to improve implementation during upcoming phases. 
This enables a structured review of implementation prior to phase close-out in Step 9 to 
evaluate the effectiveness of risk assessments and security practice implementation. 
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4. LESSONS LEARNED FROM SELECTED PROJECTS 


This section contains the results of Task 2 documenting the lessons learned from projects 
selected for implementation of the security practices. The Task | statistical analysis 
provided the desired set of characteristics for industry exemplary projects to assist in 
identifying projects for implementation of the security practices. With knowledge of 
characteristics of projects desirable for implementation analysis, CII sought from its 
membership projects to serve as case studies for documentation of lessons learned. 
Section 4.1 describes the methodology used to obtain projects that would produce both 
quantitative and anecdotal information as the basis for lessons learned. Section 4.2 
provides a description of the selected case study projects and Section 4.3 documents the 
lessons learned to include perceived benefits from implementing the project security 
practices. 


4.1 Task 2 Methodology 


Early 1n the research, CII distributed a one-page document requesting industry 
participation and outlining the case study approach. The request was disseminated to 
companies with projects likely to meet the desired exemplary project characteristics 
desirable for the research. The request for participation outlined assistance required in 
the gathering and documentation of security practice implementation issues. Minimal 
restrictions were placed on the completion status of prospective projects, but the 
solicitation document suggested that projects in the front-end planning and design phases 
would prove ideal due to the majority of security practices extant in those stages of 
project development. 


In an effort to garner support for the research, CII highlighted the following benefits to 
participation in the study: exposure to and understanding of the security best practices, 
access to knowledge captured within the original research, and the opportunity to 
contribute to improved industry security through the identification of lessons learned in 
the implementation of the security best practices. The document also confirmed CII’s 
strict adherence to policies of confidentiality and provided participants the opportunity to 
review the case study findings prior to publication. 


Following dissemination of the request for participation, CII carefully followed up with 
prospective organizations, a step which proved critical to obtaining participation. 
Conference calls were scheduled with interested teams. The purpose of the initial 
conference call was to inform the project team of the planned approach for the case 
studies, provide an initial orientation on the use of the security practices, and to identify 
issues the project team may have had concerning their participation. The conference call 
provided the project teams with a background on the original NIST and CII research on 
the security practices and an overview of the nine-step implementation process. A 
specific goal of the call was to assure project teams that security practice implementation 
required no formal training and simply involved integrating security practices into 


2) 


existing project team processes. In addition, the conference call served to assure teams 
that: the process was not an assessment of security posture, but rather of security practice 
implementation, that company and project confidentiality would be maintained, and that 
specific project data from the study were available only to qualified CII staff. Following 
the conference call, a face-to-face meeting was scheduled. 


A purpose of the face-to-face meeting was to provide project teams a more detailed 
orientation on the use of the security practices. Expected outcomes of the meeting 
included team familiarization with the security practices, the nine-step implementation 
process, and available implementation tools (SRI Tool). A secondary purpose of the 
meeting was to capture initial feedback that could result in future lessons learned. The 
agenda for the meeting outlined the sequence of events, estimated time requirements, and 
event description as given in Appendix D. During the meeting, facilitated evaluations of 
the project threat and consequences of a security breach were conducted and the team 
answered security practice questions for their specific project. Project team responses to 
the security practice questionnaire were used to generate an SRI score using CII’s secure 
web-based SRI Tool. To assist project teams with an understanding of the practices and 
the implementation process, a reference card included in Appendix E was developed and 
distributed to team members. The four page document summarizes the security practices, 
the nine-step process for implementation, and the threat and consequence matrices. Team 
members were also provided a link and password to access CII’s website for access to the 
SRI Tool and numerous references as additional background material. At the conclusion 
of the meeting, all participants agreed upon a path forward to continue security 
implementation and to gather feedback for potential lessons learned. A point of contact 
for each project team was identified to facilitate discussion and provide data for entry into 
the SRI questionnaire producing a score for their project. 


Follow-up with each project team was initially made via email and telephone. If the 
project team indicated success with implementation of the security practices and use of 
the SRI Tool, no additional face-to-face meeting was scheduled and all subsequent 
feedback to include SRI scores and lessons learned were obtained by telephone or email. 
However, if a project team expressed interest in additional hands-on facilitation or 
expressed difficulty in practice implementation and SRI scoring, a second face-to-face 
meeting was scheduled with key project personnel. 


The second meeting permitted additional facilitation in use of the practices, 
documentation of challenges to implementation, and the gathering of data on security 
practice implementation benefits. The second meeting afforded a more detailed 
understanding of the challenges and benefits realized by the project team during security 
practice implementation and assisted with the documenting of lessons learned for future 
implementation. The agenda for the second meeting is provided in Appendix F. 


From the face-to-face meetings and email and telephone updates, project teams provided 
sufficient information to document lessons learned and SRI scoring data. Although the 
documentation of lessons learned was the primary objective of this research, SRI data 
were also analyzed to determine statistical differences in scores from this research and 
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scores gathered during the original research in 2004. Analyses of means and variances 
were performed for SRI scores of projects with similar threat and consequence levels to 
assess differences in scores resulting from the facilitated approach to use of the practices 
implemented in the case studies. After assessing differences in the two data sets, the new 
data were added to the original data to update statistical models relating SRI scores to 
consequence levels for a given level of threat. These models are documented in a 
doctoral dissertation produced in conjunction with the original research (Sylvie, 2005). 


4.2 Description of Selected Projects 


As a result of the project selection process, three projects from separate organizations 
were chosen for analysis providing a mixture of public and private sector projects from 
the petro-chemical and transportation infrastructure sectors. However, shortly before the 
face-to-face meeting with the public sector organization, hurricane Katrina occurred 
disrupting the transportation infrastructure project and forcing a change in the projects 
available for analysis. The final set of projects analyzed included four chemical and 
petro-chemical projects from two private companies. The four projects afforded analysis 
of implementation data and SRI scores for nine phases of project life cycle. Similar to 
the original research, all projects were from the heavy industrial sector with an emphasis 
on chemical manufacturing and petro-chemical processes. 


Although actual cost data were provided for only one of the four projects, estimates from 
the scope of projects place two of the projects within the over $50 million cost category 
and the remaining two projects in $15 million to $50 million cost category. Fortunately, 
the projects consist of both the addition and grass roots classifications providing the 
opportunity to observe implementation of the security practices over both of these project 
types. This was particularly important since the issues of security practice 
implementation vary considerably among the different classifications. Even though all 
analyzed projects were domestic, each came from differing regions within the United 
States from the East Coast and Southeast to the Midwest and Gulf Coast areas. The 
geographic dispersion of the projects across the country afforded the opportunity to 
observe security practice implementation under varying circumstances and against a 
range of potential threats. 


Participating project teams consisted of project-level personnel with either direct or 
indirect organizational (corporate) level security input. Indirect input includes those 
resources that were available to the project team, but not assigned to the specific project. 
While most project teams had indirect organizational input, one team included the direct 
services of their corporate Health, Safety, and Environmental specialist whom assisted in 
all discussions. A majority of the teams were composed of project-level personnel 
including project managers, construction managers, and project engineers with two to 
four members participating per project. 


One project team incorporated significant contractor input during all interviews and SRI 
score assessments. This greatly assisted the research by creating a broader perspective on 
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the project team’s ability to implement the practices which enhanced the lessons learned 
from the implementation process. 


Members of the project teams participated in a facilitated SRI scoring session for their 
projects producing the nine phase SRI scores presented in Table 4-1. The projects are 
identified as A through D to maintain the confidentiality of the projects. The nine phase 
SRI scores are categorized by project, threat and consequence level, and phase of project 
execution. Scores are provided on a 0 to 10 scale for the phases shown. Calculation of 
SRI scores is described in detailed in NIST GCR 04-865 (Thomas, et al., 2004). 


Table 4-1. Case Study Phase SRI Scores 


Score 


a 
D PS [eS A PE 


The nine project phases for which scores were assessed from the four selected projects 
consist primarily of the early project phases of front-end planning (FEP) and design. 
This is fortunate since security practices implemented in these phases have more cost- 
effective impacts on project security as determined in the NIST and CII research. Out of 
the 79 total questions (33 with replications for project phases) within the SRI 
questionnaire, over 60 percent survey practices that are implemented prior to the 
construction phase. In addition these practices are weighted more heavily than practices 
implemented in the later project phases (Thomas, et al., 2004). 


As in the original research on the security practices, all data analysis is performed using 
phase SRI scores rather than project scores due to the relatively small data set. Also, only 
one of the four projects had completed all five project phases due to the relatively short 
duration of the case studies in comparison to estimated project completion time and a 
desire to study teams in the early phases of planning and design. Current, rather than 
completed projects, better provided an opportunity to observe project teams 
implementing the security practices and their challenges to implementation. 


Seven of nine SRI scores in this research come from projects assessed at a Threat Level 2 
as compared to a majority coming from a Level | assessment in the original research. 
This may be due to the facilitated approach in arriving at a consensus threat assessment 
or it may be the result of the type of projects targeted for this research; specifically the 
industrial projects thought to be at greater risk. Three of the four case study projects 
reported worst case Consequence Levels of 4 with the other reporting a level of 3. This 
also distinguishes this data set from the original where half of the projects reported 
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Consequence Levels of | or 2 and nearly as many reported Level 3 as Level 4. Again 
either the facilitated process of establishing ratings or the potentially greater consequence 
of security breach for chemical and petro-chemical projects could be the cause of the 
differences. The higher consequence levels of this research are consistent with the type 
of projects used in the case study. 


Two of the projects in this research used the traditional design-bid-build project delivery 
process. The other two projects used the design-build and multiple design-build delivery 
systems. The distribution of project delivery systems in this research is similar to that of 
the original research. 


4.3. Lessons Learned Findings 


Previous sections presented the methodology followed in this research and described the 
projects selected for case study analysis for the development of information on the 
implementation of security best practices. This section documents the lessons learned 
from the implementation of these practices and identifies specific recommended 
modifications to the implementation process and SRI Tool based on project team input. 


The opportunity to observe project teams during the implementation of the security best 
practices and use of the SRI Tool produced a considerable amount of information on the 
project team’s ability to implement the practices, benefits perceived by the teams from 
use of the practices, and barriers to practice implementation. This information 1s 
captured as lessons learned in the following four sections: Section 4.3.1 documents 
difficulties that project teams experienced in achieving consensus threat and consequence 
assessments and provides a recommended technique to assist in achieving this consensus, 
Section 4.3.2 identifies suggested changes to the CII web-based SRI scoring tool to 
improve usability and increase project team comfort with the system, and Section 4.3.3 
discusses benefits of security practice implementation. Finally, Section 4.3.4 provides 
SRI score analysis using data collected in this research and the original NIST and CII 
research to assess impacts on the SRI scores of changes to the security best practices 
implementation process developed through these case studies. 


4.3.1 Techniques for Evaluating Threat and Consequences 


Figure 3-2 depicted the nine-step process for implementation of the security best 
practices as defined in the CII Implementation Resource IR BMM-3 (CH, 2005). Step 2 
of the process requires the development of an activity risk matrix to assist the project 
team in selecting the security practices most relevant to the project. Prior to developing 
the risk matrix, the implementation resource suggests that the project team assess the 
threat and consequence levels for their project to better enable the team to develop the 
risk matrix. Project teams participating in the case studies; however, experienced 
difficulty in reaching a consensus on threat assessments and to a lesser extent, 
consequence assessments. Observation of the teams revealed that team personnel, 
especially when supported by organizational security specialists, possessed the necessary 
experience and technical knowledge to accurately assess the threat, but the lack of a 
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systematic assessment process for teams to follow led to difficulties in achieving 
consensus. 


To address this problem, a four-step process was developed with project team input to 
systematically address the threat from potential adversaries and better achieve a 
consensus on threat assessment. These steps included: 


Identify likely potential adversaries. 

Assess each adversary’s capability to conduct an attack on the project. 
Assess each adversary’s intent to attack the project. 

Look at current and historical events to identify attacks on similar projects 
by these adversaries. 
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Once likely threats were identified, teams assessed the capability of the adversary to 
attack their project. This capability was assessed from characteristics such as technical 
and tactical competence and access to resources. Assessment of an adversary’s intent 
required analysis of the reasons for conducting an attack and an adversary’s potential 
gains from the attack. A review of historical data on attacks for similar projects 
provided the project team an additional perspective on capabilities and intent. 


As the teams considered each adversary, they recorded their assessments in a matrix 
format similar to Table 4-2. In addition to adversary, capability, intent, and evaluation of 
historical events, elaborating remarks, and threat level assessments using the criteria and 
scale from Table 3-2 were recorded. The 4-step process structured the identification and 
evaluation of individual threats and by organizing these data, the team was better able to 
arrive at a consensus threat level rating. Using these steps, project teams established a 
comprehensive list of credible, potential threats to facilitate an overall project threat 
assessment. For the example shown in Table 4-2 below, the team determined that the 
most credible threats for their project included a disgruntled worker and an activist 
seeking press. By focusing on these specific threats, the team could better arrive at a 
consensus assessment for the overall project threat rating. As shown by the bolded fields 
in the table, assessments for the disgruntled worker and activist seeking press drove a 
consensus threat rating of 4. 


Table 4-2. Threat Level Assessment Matrix 


* Recent 
High population and profile 
Terrorist attack High Low Low targets not similar to current 2 
project 
Disgruntled : ' Similar events occurred in the 
Activist sniper Ritcereis roe et Little intent to endanger others or 
themselves 
Activist : ; : Recurring events confirm high 
seeking press level of threat for this action 
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Although project teams experienced less difficulty in reaching a consensus on 
consequence assessments, a similar systematic process was devised to assist with the 
process. Project teams answered specific questions about the worst case scenario 
resulting from an attack from the adversaries identified during the threat level 
assessment. The consequence assessment used the criteria and scale provided in Table 3- 
3 to assess consequences attributed to each adversary. 


Again a 4-step process, listed below, was used to address each of the criteria developed in 
Table 3-3 to assess the worst case scenario of an attack and a matrix was used to record 
and organize assessments. The steps of the process are shown below: 


1. Identify severity and location of incident: injury or fatality, on-site or off- 
site. 

2. Assess potential for environmental impact: none to extensive. 

3. Determine expected property damage: none to extensive. 

4. Identify likely business interruptions: little to very long term. 


Depicting these assessments in a matrix format as before provided the project teams with 
a means to more easily compare consequences of security breaches from each threat and 
assisted in determining the overall consequence rating due to the worst case consequence. 
Table 4-3 depicts an example consequence level assessment matrix with the overall 
project assessment highlighted in bold italics. 


Table 4-3. Consequence Level Assessment Matrix 


saat Environmental Property Business Consequence 
Offsite fatality; SneaneTE Extensive Extensive Very long term 5 
onsite fatalities 
Any offsite injury; a pe 4 ea 


No offsite injuries; 

Widespread onsite Moderate Moderate Medium term 

injuries 

Minor onsite injuries None aI Little / None 
None 


In this example, a project team assessed the possibility of injury, environmental impact, 
and property damage all at a Consequence Level of 3 using the criteria and scale of Table 
3-3. Only a short term worst case business interruption received a Consequence Level 2 
assessment; therefore, the project received an overall Consequence Level 3 assessment. 


By addressing threat and consequence assessments in this manner, project teams were 
more able to develop the activity risk matrix as part of the nine-step implementation 
process. The activity risk matrix documents security risks categorized by security 
element (physical, personnel, or information) and the measures selected to mitigate the 
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security risk (CII, 2005). The brainstorming process used by the project team to arrive at 
the threat and consequence level assessments enabled the group to identify a more 
complete list of security risks and thus, a more complete activity risk matrix. 


4.3.2 Modifications to Web-based Scoring Tool 


The nine-step implementation process discussed in Chapter 3 encourages project teams to 
use the CII web-based questionnaire (SRI Tool) to record their threat and consequence 
levels and to assess their level of security practice implementation. Assisting the case 
study project teams with the use of this tool provided excellent general feedback and two 
specific instances for improving the web-based tool. 


First, project teams confirmed their ability to effectively use the SRI Tool to assess 
security practice use and to generate an SRI score quantifying the level of use. The ease 
of system use allowed teams to conduct multiple assessments with interim project phase 
scoring sessions. This repetition, in turn, enabled teams to become more familiar with 
the practices themselves and ultimately reduced the total time to complete an assessment 
and scoring session. Project teams indicated no difficulties in understanding and 
answering all 33 security practices within the questionnaire. This finding 1s particularly 
important since the tool was designed for use with no formal training. One suggestion; 
however, was identified to improve questionnaire clarity by ensuring that project teams 
properly framed each question with “Security was a consideration in:” followed by the 
activity where security should be incorporated. When this was done project team 
members found it easier to answer the Likert-type scaled responses. 


Figure 4-1 provides a suggested reorganization of information provided on the computer 
screen to ensure that respondents frame the questions with the lead-in phrase “Security 
was a consideration in:” as they answer the questions. By placing the lead-in phrase 
closer to the phrase that describes the practice: in this case, “establishing project 
objectives” the intended question becomes clearer and therefore, the response should be 
of higher quality. Also, the project ID number and name should be moved away from the 
question to allow the respondent to focus further on the question being asked. 
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Figure 4-1. Suggested SRI Questionnaire Modification 


Front-End Planning 


. Security was a consideration in: 


Gj Project ID: HLS1092 Graduate Research 2005 (Ramey 


A establishing project objectives 


| Examples: 


e rehabiity and operating philosphy 
e aftordabilty and feasability 

e constructability 

e future expansion 


Send us your thoughts. 


Figure 4-2 depicts a minor but important modification to the threat assessment screen for 
the front end planning project phase. Project teams recommended changing the 
“Submit” button at the bottom of the web pages to a “Continue” button. Although the 
“Submit” button merely saves the page data and continues with the questionnaire, this 
was not clear to the teams and they thought that clicking on the button actually submitted 
data to CII before they were ready to do so. Of importance to project teams 1s their 
ability to maintain control of all SRI data until they make a decision to submit it to CII. 
Changing the button from “Submit” to “Continue” will remove any ambiguity and 
eliminate a potential barrier for project team use. 
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Figure 4-2. Web-based SRI Threat Assessment 


Front-End Planning Phase Threat Selection 


Select the Threat Level for the Front-End Planning Phase: 


Project ID: HLS1092 Graduate Research 2005 (Ramey) 


Indicates that a definite threat exsts against the asset and that the adversary has both the 
© Very High —s capability and intent to launch an attack or commit a criminal act, avd that the subject or 
‘similar assets are targeted on a frequently recurnng basis. 


— : — 
Indicates that a credible threat exsts agaist the asset based on knowledge of the 
‘adversarys capability avd tent to attack or commit a criminal act against the asset, based 
‘on related incidents having taken place at sumilar assets or i similar situations. 


Indicates that there is a possible threat to the asset based on the adversarys desire to 
‘compromise similar assets and/or the possibility that the adversary could obtain the 
capability through a third party who has demonstrated the capability in related incidents. 


Indicates that there is a low threat against the asset or similar assets and that few known 
adversaries would pose a threat to the asset 


Indicates no credible evidence of capability of intent and no history of actual or planned 
threats against the asset or simular assets. 


Overall, project teams confirmed their comfort with and the ease of use of the web-based 
SRI Tool. Use of the tool provided an additional benefit of team security awareness. 
Responding to the security practice questionnaire assisted project teams with identifying 
aspects of project security often not considered previously. The questionnaire, as 
intended, served as an effective security checklist. 


4.3.3 Benefits of Security Practice implementation 


Probably the most significant finding identified through the case studies is the recognition 
that the security practices and SRI Tool provide project teams access to a flexible and 
structured framework for the integration of security early into the project delivery 
process. This framework proved applicable to projects of varying characteristics such as 
grass roots or additions. Project teams identified particular benefits of implementation of 
the practices on projects when companies have no formal or only weak security 
procedures. This attribute was noted by a contractor; because contractors work with a 
wide array of owner companies that may or may not have well-defined security 
procedures, the security practice implementation process provides a well-researched and 
structured process for implementing security on projects regardless of an owner’s 
emphasis on security. Project managers and engineers stated that the implementation 
process was flexible and easily adaptable to projects, even where the organization had 
formal, top-driven security procedures. 


Two of the four case study project teams reported that their companies imposed a top- 


down driven security process managed by corporate security personnel. Initially, these 
teams expressed concern that their directed procedures dictated their approach to security 
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implementation and that this would ultimately reduce the effectiveness of or the need for 
the NIST and CII security practices and nine-step implementation process. A team even 
stated that the existence of an organizational security process often precluded them from 
integrating security into the project delivery process at the project level. After additional 
discussion; however, the team concluded that the NIST and CII procedures for security 
practice use did not conflict with corporate guidance, and even actually increased their 
involvement with security implementation on the project. 


The practices provided the teams with a structured process that could perhaps even assist 
them in complying with company security requirements. The flexible nature of the SRI 
Tool could be used to quantify use of security practices directed by corporate security 
programs as well as practices identified in NIST GCR 04-865. Since the practice and 
SRI Tool were developed for project team use, they can actually assist in getting the 
project team involved with security, which is a particular benefit for companies that 
manage security at the corporate level. The SRI Tool provides the team with qualitative 
and quantitative feedback that they can share with corporate security managers and it can 
also assist with the identification of redundant requirements. Comparison of output 
across multiple projects provides companies a new perspective on security awareness and 
implementation. 


The timing of practice implementation forces companies to consider security early in the 
project lifecycle when it can be addressed in a more cost-effective manner. As noted in 
NIST GCR 04-865, the Security-Influence Curve depicted in Figure 4-3 indicates a 
greater ability to influence security 1n a cost-effective manner in the early planning 
phases of a project than during the execution phases. Security add-ons later in the 
construction and start-up phases tend to produce cost overruns and schedule impacts and 
almost always ensure higher security costs. Recommended security modifications based 
on the project team input early in the planning phases of the project will reduce the 
amount of change orders needed within the construction phases of a project. 
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Figure 4-3. Security-Influence Curve 
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Case study findings also suggest a benefit to using the security practices on recently 
completed projects as well. The flexibility of the SRI Tool and the nature of the data 
collected permit assessments as part of project post-mortems. For example, one project 
team using the tool on a recently completed project identified a need to improve the 
screening of contractor and subcontractor employees and delivery personnel for an 
appropriate level of clearance. After evaluating this project, the team felt that they did 
not adequately plan for security for subcontractor personnel assisting with startup 
procedures. That project team experienced difficulty when screening these workers late 
in the construction and early in the startup phases of the project. By identifying this 
deficiency as part of a post-project review, the project team emphasized the screening of 
startup subcontractor personnel early in the startup planning for an ongoing project, 
resulting in a higher level of security awareness and more effective implementation. 


4.3.4 SRI Score Analysis 


The original research hypothesized that a meaningful relationship could be established 
between threat level, consequence level, and SRI scores, yet it stopped short of 
establishing these relationships do to a lack of data available at the time. The 
establishment of a statistically significant relationship could prove valuable in 
determining norms for security practice use for various levels of threat and consequence 
(Thomas, et al., 2004). Although the original research documented in NIST GCR 04-865 
did not establish the relations between threat, consequence, and SRI score, preliminary 
relationships were provided in the academic research conducted in conjunction with the 
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research and these are documented in the dissertation of the graduate research assistant, 
Jon Sylvie (Sylvie, 2005). It is useful to determine if the data collected in the case 
studies research can be combined with the original data for further analysis of these 
relationships. The following paragraphs compare the scores received during the case 
studies with those generated during the original research conducted by CII. Also, the 
original models documented in Sylvie’s work establishing the relationships between 
consequence level and SRI score for a given threat level are updated to assess the impacts 
of the new data. 


4.3.4.1 Differences in Case Studies and Original SRI Scores 


The ease of use of the web-based tool contributed to case study project teams completing 
nine phases of questions producing SRI scores for each phase. Although complete 
project SRI scores to include scores for each project phase would be desirable, analysis of 
phase data is acceptable as discussed in section 4.2 and NIST GCR 04-865 (Thomas, et 
al., 2004). Table 4-4 presents the SRI scores for each phase scored for each of the case 
study projects. 


Table 4-4. Case Study Phase SRI Scores 


aaa ect 


Procurement 
Construction 


As shown in the table, seven of the nine phase scores were from projects with Threat 
Level 2 and Consequence Level 4. The original research produced eleven phase scores 
with Threat Level 2 and Consequence Level 4. For the most meaningful comparisons, 
these seven case study scores are compared to the eleven original study scores with the 
same threat and consequence levels for analysis of differences in means and variability. 
These phase score comparisons are made in Table 4-5 and Figure 4-4 below. 


Figure 4-4 uses a Box-Whiskers Plot to compare the two data sets. The “whiskers” 
extend from the minimum value to the 25" percentile and from the 75" percentile to the 
maximum value for each data set. The “box” contains the inter-quartile range—the 
middle 50 % of the observations. The 25" percentile, 50" percentile (median), and the 
75" percentile are shown as vertical lines or “hinges” in the box for each data set. The 
mean value of each data set is designated by a small diamond within the box. 
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Table 4-5. SRI Score Descriptive Summary for Original Research and Case 
Studies Data (Threat Level 2 and Consequence Level 4) 


Original Research Case Studies 
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Figure 4-4. SRI Score Graphical Summary for Original Research and Case 
Studies Data (Threat Level 2 and Consequence Level 4) 


Original and Case Studies Data 
(Threat Level 2, Consequence Level 4) 


ae Case Studies Data 
Original Data 


(Sylvie, 2005) 


6.00 6.75 7.50 8.25 ; 9.00 9.75 | 
Phase SRI Score 


A comparative analysis of the two data sets reveals a striking difference. While both the 
original and case studies data have somewhat similar means and medians, they have very 
different variances. Statistical tests for differences in the means and variances are 
provided in Appendix G. These tests confirm what is apparent from Table 4-5 and Figure 
4-4, while the difference in means is not significant; the difference in variance 1s highly 
significant. Caution is appropriate though when drawing conclusions from the small 
samples available. 


A review of the differences in process for data collection between the original research 
and the case studies is useful for interpreting the differences in scores. The most notable 
difference in the data collection process was the face-to-face facilitation with the case 
study project teams as previously described. The increased interaction with these teams 


34 


had the potential to significantly affect the data received. Possible impacts include higher 
case study scores due to the increased security awareness and familiarity with the security 
practices and implementation process. This of course is not supported by the data. 
However, a better understanding by the project teams of the process for implementation 
of the security practices could result in higher quality data that more accurately reflects a 
project’s level of security implementation. The reduction in variability of case study 
scores may be an indication of this. 


4.3.4.2 Impact on Original Regression Models 


Preliminary regression models established in the academic research in conjunction with 
the original research produced relationships between threat level, consequence level, and 
the SRI score. These consisted of simple linear models regressing phase SRI scores 
against consequence levels for the levels of threat reported. This research recreated the 
regression models from the original research for Threat Level 2 projects (7 of the nine 
case studies scores) to assess the impact of the addition of the newer data. Figure 4-5 and 
Table 4-6 depict the relationship from the original research and updates it to include the 
seven scores from the case studies. A complete summary of the regression models is 
provided in Appendix G. 


Figure 4-5. Regression Model for Original Research and Case Studies Data 
(Threat Level 2 and Consequence Level 4) 


Regression Analysis With & Without Case Studies Data 
(Threat Level 2) 


10 - 
© 
8 J 
° © 
| 
@ 6- 8 Xs 
° 
° 
” 
om 4 © 
| 8 Original Data (n=24) With Case Studies (n=31) 
a1 mm y = 1.7054x + 1.0282 y = 1.7029x + 1.0335 
R? = 0.4840 R? = 0.5354 
0 +—_______ —_ oe — | 
1 2 3 S 5 


Consequence Level 


° Case Studies © Original 


30 


Table 4-6. Regression Model Statistics for Original Research and With Case 
Studies (Threat Level 2 and Consequence Level 4) 


[_________] Original Research | With Case Studies. 
[bervationsi anon ARREE 


The new model updated to include the case study observations is very similar to the 
original model; however it has been improved in all summary statistics. Figure 4-5 
reveals that the regression lines are essentially identical which can be confirmed through 
comparison of their slopes and intercepts. The slope of the original model (1.7054) is 
slightly greater than the model with case study data added (1.7029) while the intercept 
with the additional case study data (1.0335) is slightly higher than the original (1.0282). 
The most notable difference lies in the improvement of the model fit as depicted in Table 
4-6 by the R-square improvement from 0.4840 to 0.5354 and the reduction in the 
standard error from 1.7608 to 1.5424. In additional, the significance of the model 
improved from .0002 to less than .00001. While the significance would be expected to 
improve from the addition of more data, of more importance is the improvement in the 
explanatory power evident in the R-square and the increased reliability of its predictive 
power through the reduction of standard error. The improvement in the model is 
predictable from comparison of the original and case study data in Section 4.3.4.1. The 
case study data obtained through a facilitated process cluster around the mean of the 
original data and are much less variable which reduces the standard error in the 
regression models. 


36 


5. CONCLUSIONS AND RECOMMENDATIONS 


These case studies produced some significant and interesting findings from actual project 
experiences of teams implementing the security best practices developed through the 
joint NIST and CII research documented in NIST GCR 04-865. This information is of 
particular importance since an absence of such data has proved an impediment to teams 
contemplating use of the practices. Although the case studies targeted projects from the 
heavy industrial sector; specifically the chemical and petro-chemical project types, it is 
likely that the lessons learned are applicable to most industrial projects and to projects 
from the building and infrastructure sectors as well. 


5.1 Conclusions 


The case study projects provided significant supporting information confirming a project 
team’s ability to implement the security practices using the nine-step implementation 
process. The security practices and SRI tool provide project teams access to a flexible 
and structured framework for the integration of security into the project delivery process. 
Project teams experienced the most success when they incorporated implementation steps 
into existing organizational processes and procedures. Conducting project level reviews 
of security implementation during normally scheduled team meetings eliminated the need 
for additional security specific meetings and minimized the total impact on project team 
time. 


Specific lessons learned through these case studies include: 

e Project teams have difficulty assessing the levels of threats and consequences of a 
security breach for their projects without a structured process and some 
facilitation. 

e Minor modifications should be made to CII’s online questionnaire for assessing 
security practice use to enhance its usability and increase project team comfort 
with the system. 

e The security best practices and nine-step implementation process provide a 
flexible and structured framework for integrating security into the project delivery 
process. 

e Project teams identified benefits of implementing the security best practices even 
if their company had well-defined security procedures in place. 

e The SRI tool encourages project team participation in security practice 
implementation early in the project lifecycle when it is most cost-effective. 

e The security practices and the SRI Tool can be used effectively for post project 
analysis to identify lessons learned for future projects. 

e Facilitation of use of the SRI tool and nine-step process for implementation does 
not increase scores on average; however, it significantly reduces variability for 
projects with similar threat and consequence levels. 
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Lessons learned for barriers to implementation of the security practices were predictable: 
e Perceived time constraints for implementing additional practices. 
e Team buy-in for the applicability to their project. 
e A lack of understanding of the benefits of security practice implementation. 


5.2 Recommendations 


While this research confirmed that project teams possess the necessary expertise to 
implement the security practices, the recommendations outlined in this section could 
remove barriers to their implementation and increase the overall level of practice use 
industry wide. 


The Construction Industry Institute should continue to educate industry stakeholders on 
the security best practices and use of the nine-step implementation process. Continued 
industry use should affirm the flexibility of the practices for a wide range of projects and 
contribute to their validation. CII conferences and other industry forums are appropriate 
for promoting awareness of the practices and implementation process. 


CII should continue collecting SRI scores. Additional data will permit continued 
development of the relationship between expected SRI scores and threat and consequence 
levels. Additional SRI score data will permit establishment of correlations between SRI 
scores and project outcomes of cost, schedule, and safety. 


Wider dissemination of the security practices and the implementation processes to 
corporate risk and security management personnel should improve overall security 
implementation. Although the CII developed security practices to promote security at the 
project level, integration of these practices into corporate policies should improve project 
team buy-in, communications, and overall security. 


38 


Appendix A: Statistical Notes 


Confidentiality 

When there were fewer than 10 projects available in a category or when fewer than three 
companies submitted the data, no statistical summaries are provided. This is consistent 
with the CII policy on confidentiality and in such cases the code “C.T.” (confidentiality 
test) was inserted in the tables. 


Statistical Warning Indicator 

When there are fewer than 20 projects included in any table cell, an asterisk (*) follows 
the data value. This notation indicates that the data in that table cell should be interpreted 
with caution due to the small number of projects represented in that cell. 


Removal of Statistical Outliers 

Prior to performing the Task | statistical analyses, all outcome metrics values calculated 
were screened to remove statistical outliers. This step was incorporated to remove values 
so extreme that their inclusion would likely distort the statistical summaries produced. 
The technique used to identify statistical outliers was the same used to define outliers in 
most statistical texts. This is also the same definition used for outlier commonly used in 
the preparation of box and whisker plots. All values exceeding the 75th percentile value 
+1.5 times the inter-quartile range or those less than the 25th percentile value - 1.5 times 
the inter-quartile range were excluded. 
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Appendix B: Metric and Project Phase Definitions 


Performance Metric Formulas and Definitions 


Performance Metric Category: COST 


: Project Cost Growth 


: Delta Cost Growth 


: Project Budget Factor (Contractor data only) 


: Delta Budget Factor(Contractor data only) 


: Phase Cost Factor (Owner data only) 


: Phase Cost Growth (Owner data only) 


Definition of Terms 


Actual Total Project Cost: 
e Owners — 


o All actual project cost from pre- 
project planning through startup 


Exclude land costs but include in- 
house salaries, overhead, travel, etc. 


e Contractors — Total cost of the final scope 
of work. 


Initial Predicted Project Cost: 


e Owners — Budget at the time of 
authorization. 


Contractors — Cost estimate used as the 
basis of contract award. 


Formula: 


Actual Total Project Cost - Initial Predicted Project Cost 
Initial Predicted Project Cost 


Formula: 
| Cost Growth | 


Formula: 


Actual Total Project Cost 
Initial Predicted Project Cost +Approved Changes 


Formula: 
| 1- Budget Factor | 


Formula: 
Actual Phase Cost 
Actual Total Project Cost 


Formula: 


Actual Phase Cost — Initial Predicted Phase Cost 
Initial Predicted Phase Cost 


Actual Phase Cost: 


e All costs associated with the project phase in 
question. 


See the Project Phase Table in Appendix B for 
phase definitions. 

Initial Predicted Phase Cost: 

e Owners — Budget at the time of authorization. 


e Contractors —— Budget at the time of contract 
award. 


See the Project Phase Table in Appendix B for 
phase definitions. 


Approved Changes: 


e Estimated cost of owner-authorized changes. 
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Performance Metric Category: SCHEDULE 


: Project Schedule Growth 


Metric: Delta Schedule Growth 


Metric: Project Schedule Factor (Contractor data only) 


Metric: Delta Schedule Factor(Contractor data only) 


Metric: Phase Duration Factor (Owner data only) 


Metric: Total Project Duration 


Metric: Construction Phase Duration 


Definition of Terms 


Actual Total Project Duration: 
(Detail Design through Start-up) 


e Owners — Duration from beginning of detail 
design to turnover to user. 


Contractors - Total duration for the final 
scope of work from mobilization to 
completion. 


Actual Overall Project Duration: 
(Pre-project Planning through Start-up) 


e Unlike Actual Total Duration, Actual 
Overall Duration also includes time 
consumed for the Pre-Project Planning 
Phase. 


Formula: 


Actual Total Proj. Duration - Initial Predicted Proj. Duration 
Initial Predicted Proj. Duration 


Formula: 
| Schedule Growth | 


Formula: 


Actual Total Project Duration 
Initial Predicted Project Duration + Approved Changes 


Formula: 
| 1- Schedule Factor | 


Formula: 
Actual Phase Duration 
Actual Overall Project Duration 


Actual Total Project Duration (weeks) 


Actual Construction Phase Duration (weeks) 


Actual Phase Duration: 


e Actual total duration of the project phase in 
question. See the Project Phase Table in 
Appendix B for phase definitions. 


Initial Predicted Project Duration: 


e Owners — Predicted duration at the time of 
authorization. 


Contractors - The contractor's duration estimate 
at the time of contract award. 


Approved Changes 


e Estimated duration of owner-authorized 
changes. 
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Performance Metric Category: SAFETY 


Metric: Total Recordable Incident Rate (TRIR) Formula: 


Total Number of Recordable Cases x 200,000 
Total Site Work-Hours 


Metric: Dart Rate (LWCIR) Formula: 


Total Number of DART Cases x 200.000 
Total Site Work-Hours 


Definition of Terms 


Recordable Cases: All work-related deaths. DART Cases: Incidents resulting in days away 


and illnesses, and those work-related from work, restricted activity, or transfer. 
injuries which result in: death, loss of 


consciousness, restriction of work or 
motion, transfer to another job, or require 
medical treatment beyond first aid. 


Performance Metric Category: CHANGES 


Metric: Change Cost Factor Formula: 
Total Cost of Changes 
Actual Total Project Cost 


Definition of Terms 


Total Cost of Changes: Actual Total Project Cost: 


e Total cost impact of scope and project Ounce 
development changes. 


o Allactual project cost from pre-project 
planning through startup 


Exclude land costs but include in-house 
salaries, overhead, travel, etc. 


e Contractors — Total cost of the final scope of 
work. 
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Performance Metric Category: REWORK 


Metric: Total Field Rework Factor Formula: 
Total Direct Cost of Field Rework 
Actual Construction Phase Cost 


Definition of Terms 


e Total Direct Cost of Field Rework: Total direct cost e Actual Construction Phase Cost: All costs associated with 
of field rework regardless of initiating cause. the construction phase. See the Project Phase Table in 
Appendix B for construction phase definition. 
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Appendix C: Security Questionnaire 


was a consideration in 
establishing project objectives (e.g., reliability and 
operating philosophy, affordability and feasibility, 
constructability, future expansion, etc.). 


preparation of the specifications and requirements (e.g., 
civil/structural, architectural, water treatment, 
loading/unloading/storage facilities, substation/power 
sources, instrument & electrical, etc.). 


developing and evaluating design criteria (based on 
vulnerability assessment). 


design and material selection. 


developing the engineering/construction plan & 
approach. 


developing the procurement/materials management 
procedures and plans (e.g., warehousing, inventory 
control, key & lock control, hazardous materials). 


developing the pre-comm/turnover sequence/startup 
requirements/objectives. 


determining required site characteristics and location. 


preparing the permitting plan. 


developing the plot plan (e.g., layout, accessibility, gate 
configuration, etc.) - retrofit & greenfield. 
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Strongly Disagree 
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NA/Unknown 


iS) aaa eee 


Strongly Disagree 


was a consideration in 
evaluation of various personnel issues (e.g., 
education/training, safety and health considerations). 


development of a distribution matrix for document 
control (e.g., drawings, project correspondence, CAD, 
as-built documents). 


alignment concerning the importance of security issues 
identified in the project objectives. 


defining and purchasing security-related equipment with 
appropriate input (e.g., O&M, Security Manager, etc.). 


identifying stakeholders for the project team (based on 
vulnerability assessment). 


establishing priorities between cost, schedule, and 
required project features (based on vulnerability 
assessment). 


identifying and resourcing startup requirements (e.g., 
procurement, personnel, training). 


screening of the project team for appropriate level of 
clearance. 


screening of contractor/subcontractor 
employees/delivery personnel for appropriate level of 
clearance. 


identifying startup risks. 

developing/implementing startup security plan. 
developing system startup plan (reconciled with security 
plan). 


developing training plans (e.g., job site, O&M, startup). 


® 
® 
= 
D 
© 
a 
Qa 
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was a consideration in 


assessing & communicating effects from change orders. 


developing/implementing construction site security plan 
(e.g., fire protection and safety considerations, egress, 
emergency responder access, process shutdown) 


The project had a designated site security coordinator. 


developing business partnerships/alliances. 


Strongly Disagree 


NA/Unknown 


project information systems security plan (e.g. firewalls, 
wireless security, passwords, access controls). 


security breaches/incidents were routinely investigated. 


developing emergency response plan in coordination 
with local authorities. 
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Appendix D: Meeting 1 Agenda 


Purpose: Orient project team on use of practices and practice implementation 

Expected outcome: Project team familiar with the Security Practices Implementation Process, 
Security Practices, and implementation tools; Capture immediate team feedback for potential 
lessons learned 

Format: PowerPoint presentation in classroom setting; Web-based application of Security Rating 
Index 


1. Introduction (estimated time — 15 min.) 
Following a brief introduction of all case study participants, this section will provide the 
project team an overview of the case study, its purpose and objectives, and provide an 
update on any potential issues or concerns of team members. 


2. Project overview (30 min.) 


Optional time allotted for the project team to describe the project selected for the case 
study and provide any relevant project specific data at the team’s discretion. 


3. Implementation Background (/5 min.) 


Overview of the Project Security Practice and implementation process development and 
applicability. 


4. Implementation Process Overview (90 min.) 


In-depth review of the nine-step implementation process and recommendations to 
facilitate project team execution of each step (relative to a specific project phase). 


5. Security Rating Index Calculation (60 min.) 


Hands-on application of calculating a phase Security Rating Index score using the 
Construction Industry Institute secure website and project team input. 


6. Immediate Project Team Feedback (as necessary) 


Resources provided: 
The Construction Industry Institute (CID) will provide the project team with Implementation 


Resource hard copies, copies of all presentations, practice implementation graphic training aides, 
and electronic invitations to CII’s secure Implementing Project Security Practices Website. 


Resources requested: 
Conference room (or equivalent) 


Computer projection equipment and screen 
Internet access within conference room 
Method of recording data (dry erase boards, butcher block paper with markers, etc.) 
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Appendix E: Implementation Process Reference Card 


Implementing Project Implementation Resource IR BMM-3 


Security Practices Construction Industry Institute® 
Benchmarking & Metrics 


il. Review phase checklist Actimtty in which security should be considered: 
before phase start | 


Estatiahing project otsectves (e.g. rebabity and operating 
philosophy, affordabiity and feasibility, constructablity 
future expansion, etc.) 


Preparation of the specdicatars and requrements (e.g 

civil'stnuctural, architectural, w ater Yeatrent 

laacing/uninading ‘storage facities, substaton'pow er 
sources, instrument & siectrical, et ) 


2. Develop activity risk 


matrix 
Project Name: Chemical Plant B if 
Project Phase: Construction 
Risk Type (Phys, | 
Risk Pers, Info)' Measures to address risk 


Conduct background check of all employees 

Ensure vendors submit verification of background check 
for all employees 

Establish 100% identification requirement for site 


3. Identify security 


; Equipment Theft Bereonnel! Require written authorization to remove tools/equipment 
d v Vv / ! 
practices relevant to Information [pede 
project phase Establish emplyee parking area off site 
Ensure entrances to site 100% secured 
Conduct background check of all employees 
ec dante Wieie Train supemsors to recognize signs of drug/alcohol 
Drug/Alcohol Personnel abuse 
pennies co Information Conduct random drug screenings 
Require employees to show prescriptions for any drugs 
4. Implement practices as ——— prougiion sae 
appropriate 
5. Complete questionnaire Start-up 
& calculate phase SRI = 
Security wat a Consideration wi 
score 1451070 4 Tere 


6. Conduct periodic review 


7. Update phase SRI 
score 


Security - Influence Curve 


Front-End 
Planning 


8. Conduct post phase 
implementation review 


Cumulative Project Cost 


100% 
= 
2 
= 
=] 
9 
oo 
7) 
i] 
oO 
Cc 
® 
3 
é 
£ 
° 
2 
= 
2 
< 
0% 


Project Time oem 


9. Closeout phase SRI 
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The Security Best Practices 


Security Element?” 
Activity in which security should be considered: 


Establishing project objectives (e.g., reliability and operating 
philosophy, affordability and feasibility, constructability, future 
expansion, etc.) 


Preparation of the specifications and requirements (e.g., 
civil/structural, architectural, water treatment, 
loading/unloading/storage facilities, substation/power sources, 
instrument & electrical, etc.) 


Developing and evaluating design criteria (based on vulnerability 
assessment) 


Developing project scope Poe x a 
[5 [Design andmaterial selection EE XTX] OP XT 
See nek 


Developing the procurement/materials management procedures 
and plans (e.g., warehousing, inventory control, key & lock control, 
hazardous materials) 


SE ee 0 


Bi Developing the pre-comm/turnover sequence/startup el<( Ho 
requirements/objectives 

I Ei 
Ft en —— oc 
[12 fPevaietereminaven Cd | || TE Xx] 


Developing the plot plan (i.e., layout, accessibility, gate 
configuration, etc.) - retrofit & greenfield 


Evaluation of various personnel issues (e.g., education/training, 
safety and health considerations) 


Development of a distribution matrix for document control (e.g., 
drawings, project correspondence, CAD, as-built documents) 


The project team was in alignment concerning the importance of 
security issues identified in the project objectives. 


Security-related equipment was defined and purchased with 
appropriate input (e.g., O&M, Security Manager, etc.) 


Identifying stakeholders for the project team (based on vulnerability 
assessment) 
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Establishing priorities between cost, schedule, and required project Xx X X X x 
features (based on vulnerability assessment) 
Identifying and resourcing startup requirements (e.g., procurement, x X x 
personnel, training) 
Screening of the project team for appropriate level of clearance i 

Screening of contractor/subcontractor employees/delivery X x X x 
ict lll for eile level of clearance 
Developing/implementing startup security plan ieee X Pe ee 


Developing system startup plan (reconciled with security plan) Sie aE 


[a [oesconavaringpie eo mate Omvanm |x| xx] xx] x |x| 
_— fs 


Developing/implementing construction site security plan (e.g., fire 
28 |protection and safety considerations, egress, emergency responder 
access, process shutdown) 
The project had a designated site security coordinator | a x Fe ck Ea Pe] 
Developing business partnerships/alliances Bete A [ae] 
34 Project information systems security plan (e.g., firewalls, wireless 
security, passwords, access controls) 
Security breaches/incidents were routinely investigated = | 


Developing emergency response plan in coordination with local 
authorities 


'Phase definitions are provided in Appendix B. Phase abbreviations are as follows: 
e FEP: Front End Planning 

D: Design 

P: Procurement 

CON: Construction 

SU: Start-Up 


Activity in which security should be considered: 


x< 


x< 


x< 


*Security Elements Definitions (Center for Chemical Process Safety, 2002): 
e Physical security considerations include equipment, building and grounds design, and security 
practices designed to prevent physical attacks on facilities, persons, property, or information. 
e Personnel security includes practices and procedures for hiring, terminating, and addressing 
workplace issues; screening or background checks of employees. 
e Information security refers to practices and procedures for protection of documents, data, 
networks, computer facilities, and telephonic or other verbal communication. 
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Threat Levels 


score | tov | asinton 


Indicates that a definite threat exists against the asset and that the 
adversary has both the capability and intent to launch an attack or 
commit a criminal act, and that the subject or similar assets are targeted 
on a frequently recurring basis. 


Very High 


Indicates that a credible threat exists against the asset based on 
knowledge of the adversary's capability and intent to attack or commit a 
criminal act against the asset, based on related incidents having taken 
place at similar assets or in similar situations. 


Indicates that there is a possible threat to the asset based on the 

3 Mediu adversary's desire to compromise similar assets and/or the possibility 
that the adversary could obtain the capability through a third party who 
has demonstrated the capability in related incidents. 

2 awe Indicates that there is a low threat against the asset or similar assets 
and that few known adversaries would pose a threat to the asset. 

4 Went Ou Indicates no credible evidence of capability or intent and no history of 

2 actual or planned threats against the asset or similar assets. 


Consequence Levels 


Possibility of any offsite fatalities; possibility for multiple onsite fatalities 
Extensive environmental impact onsite and/or offsite 

Extensive property damage 

Very long-term business interruption/expense 


Possibility of any offsite injuries; possibility for onsite fatalities 
Significant environmental impact onsite and/or offsite 
Significant property damage 

Long-term business interruption/expense 


No offsite injuries; possibility for widespread onsite injuries 
Moderate environmental impact onsite and/or offsite 
Medium 
Moderate property damage 


Medium-term business interruption/expense 


Possibility for onsite injuries 

Minor environmental impact onsite only 
Minor property damage 

Short-term business interruption/expense 


Possibility for minor onsite injuries 

No environmental impacts 

Little/No property damage 

Little/No business interruption/expense 


Implementing Project Implementation Resource IR BMM-3 | 
Security Practices Construction Industry Institute® 
Benchmarking & Metrics 


Appendix F: Meeting 2 Agenda 


Implementing Project Security Practices 
Meeting 2 Agenda 


Purpose: Gather data on security practice implementation benefits and issues; facilitate use of 
security practices on current project 

Expected outcome: Understanding of all benefits and challenges realized during security practice 
implementation; Document lessons learned for future implementation; Implement security 
practices during transition to design phase 

Format: Hard copy PowerPoint presentation in conference room; Web-based application of 
Security Rating Index 


7. Introduction (estimated time — 15 min.) 
Following a brief introduction of all case study participants, this section will offer a 
review Meeting 1 and provide any new project team personnel with an overview of the 
security best practices and their implementation 

8. Practice Implementation Assessment (30 min.) 
Identify perceived benefits or challenges associated with implementing the project 
security practices on the current project. Determine causes for identified challenges and 
provide recommendations for practice improvement. 

9. Project Specific Characteristics (15 min.) 
Identify project specific risks and controls associated with job-site unique characteristics. 

10. Implementation Process Application (/20 min.) 
Facilitate / Review the implementation of the project security practices from the planning 
phase; apply the nine-step process to facilitate security practice implementation during 
the transition into the design phase. 


11. Security Rating Index Calculation (60 min.) 


Calculate phase SRI score for completed planning phase; Determine challenges 
associated with answering security practice questions as written. 


12. Project Team Feedback (as necessary) 


Resources provided: 
The project team will receive a copy of this agenda, all presentations, and practice 


implementation graphic training aides used during practice implementation. 


Resources requested: 
Conference room (or equivalent) 


Internet access within conference room (for SRI calculation) 
Method of recording data (butcher block paper with markers, etc.) 
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Appendix G: Statistical Analysis 


Analysis of Differences for Original & Case Studies Data 


—_— 
—™ |— 


ummary stats for two samples 
Original 


7.66 7.837 


1.48 


Sample sizes 
Sample means 
Sample standard deviations 


oO 
© 
wo 
ro) 
—_ 


est of difference=0 versus two-tailed alternative 
Hypothesized mean difference 


Sample mean difference -0.176 
Pooled standard deviation at OS 


Std error of difference 
Degrees of freedom 
t-test statistic 

p-value 


: 


est of equality of variances 
Ratio of sample variances 


0-value 


mY 


F 


S iF Significance F 
63.98592585 | 63.98593 | 20.63875 0.000160148 


68.20619392 | 3.100282] si 
132.1921198| #| | 
Faeivees | eae 


ae 
| a 
a 
| ae 
| ae 
ee 
a 
aa 
ei ae 
Sa 
ee 
Ren 
a he | 
Beat | 
mae | 
Lower 95% 
-1.423405748 
0.375395642 0.926895273 
ate. ot | Sa 
jot os] Calne 
eS oe 
ae) ae | es 


aes) | ae 

aa _ ae 
Fae 

R Square ae 

Adjusted R Square Fare |. 
aearE| 
Exc 

ae 

a 


Significance F 


29| 68.99084367 | 2.378995 
3 


o| 1485031888 | | 
eee cies) Lower 95% _| Upper 95% 
0.989740378 | 1.044184 | 0.305027 | _ -0.99077574 | 3.057716916 

1.100463132 | 2.305333103 


2 
1 |  79.51234514 | 79.51235 | 33.42267 | 0.000002907| Cid 
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